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PERFORMANCE OF SEVERAL CONVOLUTIONAL AND BLOCK CODES 


WITH THRESHOLD DECODING 
By Frank Neuman and Dale R. Lumb 
Ames Research Center 


SUMMARY 


The performance of several codes applicable to space communications telem- 
etry links was evaluated. The study was limited to high rate and relatively 
short constraint length codes. Explicitly, the (15,7) and (73,45) block codes 
and the (24,12) and (44,22) convolutional codes were investigated. Two types 
of threshold decoding schemes, derived by Massey, were applied, namely, major- 
ity decoding and the more powerful but complex a posteriori probability (APP) 
decoding . 

The gauss ian data channel and the decoders were simulated on a general 
purpose computer. The results show that, for the codes considered, APP decod- 
ing has approximately a 1.5 dB advantage over majority decoding. The most 
powerful code studied, (73,45) with APP decoding, gives a 2 . 1 dB improvement 
over a seventh bit parity check code. Also, new error deletion schemes were 
designed and tested for the codes studied. Because of their constant computa- 
tion rate, these codes are candidates for high data rate channels. For the 
low data rates required for deep space missions such as Pioneer, however, the 
code performances reported here form a basis of comparison with the more 
powerful sequential decoding of convolutional codes, a variable computation 
rate decoding technique. 


INTRODUCTION 


The purpose of this investigation was to evaluate the performance of 
several cyclic block and convolutional codes with different decoding tech- 
niques introduced by Massey in 1963 (ref. 1) . These techniques are called 
majority threshold decoding and a posteriori probability (APP) threshold 
decoding. 

Although for the gauss ian channel an improvement in performance was 
expected for APP decoding over majority decoding, no data were available on 
the amount of improvement possible with this more complex decoding scheme. 
Before this investigation, only the gross behavior of the codes was known for 
threshold decoding. The detail was insufficient to allow a deep space commu- 
nication link designer to decide on the application of these codes. For this 
application of coding, it was found that the measures of performance given in 
the literature, namely, block and bit error probability estimates (for block 
codes) and probability to first error (for convolutional codes), were insuf- 
ficient for comparing codes. A unifying measure is proposed in the section on 
measures of performance and has been applied in this report. 



The codes were studied in somewhat greater detail than is required to 
calculate performance figures for several reasons: first, to present suffi- 

ciently detailed information to the spacecraft communication designer to allow 
him to choose a code suitable for his particular needs, or even to reject 
these coding schemes; second, to permit the study of additional processing 
steps that might improve the performance of the individual codes beyond that 
of the decoding procedures. Some improvements are indicated in this report, 
along with the results of trial experiments. 


CODES AND DECODING TECHNIQUES INVESTIGATED 


The codes investigated were from the class of block and convolutional 
codes which can be decoded by threshold decoding. In threshold decoding, the 
value of each bit is decided by comparison with a predetermined probability 
threshold. This type of decoding is relatively simple to implement compared to 
more optimal strategies. The convolutional codes that are threshold decodable 
have been constructed essentially through trial and error techniques, hence 
the name M trial and error codes" (ref. 1) . The convolutional encoder is 
a binary shift register of length equal to the constraint length of the code. 
The shift register is tapped to calculate m-1 parity bits per information bit 
for a rate 1/m code . 

A number of cyclic block codes have been found to be threshold decodable. 
The encoder, in this case, is a tapped cyclic shift register of length equal to 
the number of information bits in a block. 

Of the codes that are threshold decodable, only relatively high rate codes 
were considered in this study. Compared to high rate codes, low rate codes 
have a lower signal-to-noise-power ratio per bit for the same information rate. 
This is not consistent with the requirement of maintaining coherent detection 
with state-of-the-art demodulation equipment. Also, the codes considered were 
of relatively short constraint length, since it is desirable for encoders for 
spacecraft application to be relatively simple. 

Threshold decoding was introduced in 1963 by Massey (ref. 1) , who 
invented two types of threshold decoding: majority decoding and a posteriori 

probability (APP) decoding of a set of orthogonal parity check equations. In 
the following section the procedure will be described briefly in terms of the 
algebraic manipulations required for the decoding process. The details of 
threshold decoding are shown by means of examples for the (24,12) convolu- 
tional code, and for the (15,7) block code in appendixes A and B. 


Majority Decoding of a Convolutional Code 

Majority decoding derives its name from the fact that a decision about 
whether to correct a received bit is based on the majority of a set of parity 
check equations. A functional block diagram is shown in figure 1. A part of 
the decoder duplicates the encoder which uses as input the received informa- 
tion bit sequence to calculate parity check bits. These are summed Mod 2 with 
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the received parity bits. The resulting equation, called an S equation, is 
a function of error terms only. An S equation will be 0 when it contains 
an even number and 1 if it contains an odd number of error terms. There are 
as many S equations as the constraint length of the code. If one assumes 
that decoding will correct all errors, the error terms in the S equation, 
which were due to the received bit before decoding, are removed by complementing 
the S registers. With this assumption, the S equations are combined into a 

smaller set of orthogonal A equations, each of which consists of the error 

term for the bit to be decoded next, as well as other error terms; but none of 
the other error terms occur more than once in the set of A equations. For 

majority decision decoding, when the majority of A equations indicates an odd 

number of errors, the decision is made to correct the bit that is being decoded. 
In equation form, the rule is expressed as 



if J 

I ^ > J/2 (1) 

i=i 

where e* is the error term of the decoded bit. The decoder incorporates an 
alarm circuit (not shown) which resets the S register when the decoder has 
attempted too many corrections in a given interval. The intention is to reduce 
propagation of errors after the decoder has made an initial error. 


APP Decoding of a Convolutional Code for the Gaussian Channel 

Majority decoding, in general, is not optimum. The equations for the 
various A^ carry different weights of evidence that there may be an error in 

i i , the bit being decoded. On the average, when A^ = 1 , the more terms the 
equation has, the smaller is the probability that the error is in ij, that is, 

ei = 1 . Therefore, optimum threshold decoding will be considered here for the 
case in which the received bits do not all have the same error probability, but 
the individual probabilities are known at the receiver. An example of such a 
channel is one employing coherent matched filter detection of a binary signal- 
ing alphabet. The matched filter output is a gaussian distributed noise volt- 
age added to the binary signal. The receiver then uses the polarity of the 
received voltage V to assign to the received bit the more probable value of 
the binary number transmitted. In addition, the amplitude of the received 
voltage can be used to compute the probability that this assignment was 
wrong . 

It is shown by Massey and is also proved in appendix C that for the time- 
varying channel, the following decoding theorem holds: 

Choose e\ = 1 

if j 

I w.A. > T (2) 

i= 1 
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Otherwise, choose 


e* = 0 


where w^ are weights and T is the threshold. Equation (2) is of the same 
form as the majority decoding theorem, where T = J/2 and w^ = 1 for all i. 
For APP decoding, the threshold and the weights are calculated for each bit 
separately from the individual bit error probabilities in the following manner: 
Define a new set of equations that corresponds to the A- in equations (2) and 
(A16) 

n 

q = I cj (3) 

j=l 

where 6 denotes either an information bit i or a parity bit p, j indi- 
cates the jth information or parity bit, depending on the superscript, and 
the summation sign denotes ordinary addition. Note that the terms correspond- 
ing to ej- are missing from the new set of equations. The Cj are calculated 
from the Bit error probabilities in the following manner: 

cj = ~log e [l - 2Pr(e?=l)] (4) 

and the weights are defined from the Ch as 

w i = 2 log e ( coth [(l/2)Ch ] } (5) 

w q = 2 log e {coth [ (1/2) c 0 ] } (6) 

where 

C 0 = -log[l - 2Pr(ei=ljJ (7) 

Then the threshold is calculated for each particular bit as 

J 

T = (1/2) l w ± (8) 

i=o 


Figures 2(a) to 2(c) are intended to clarify the above functional relationships. 
Figure 2(a) shows how, with increased energy level of the received signal plus 

noise, the individual bit error probability, Pr^ej=lj, decreases. It decreases 

faster for lower average bit error probability. However, cj increases with 
decreasing level of the received bit (see fig. 2(b)). Consequently, the sum 
(1/2) £ cj is large if many bits have below average levels. Figure 2(c) shows 
that a large (1/2) £ Cj results in a low weight, a result that agrees with 
intuition. The above generalization is illustrated by an example given in 
appendix A. 
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Majority Decoding of a Cyclic Block Code 

The decoding process for majority decoding of a cyclic block code is very 
similar to that for the convolutional code. However, the A equations are 
generated differently. After each decoded bit in a block, the indices of the 
error terms are advanced by one (modulo the block length) to obtain the A 
equations for the next bit to be decoded. The circulation of the indices in 
the decoder is the circulation or feedback of the bits in a shift register. 

The majority decoding theorem is identical to that for the convolutional code: 


J 

l A A > J/2 (9) 

i=l 


Choose 

if 


Otherwise, choose 


e\ = 0 


In the decoding shift register the decoded bits can be fed back in one of 
two ways, either as they are received or after correction. 


APP Decoding of Cyclic Block Codes for the Gaussian Channel 

The APP decoding theorem for the cyclic block code is identical to that 
for the convolutional code, namely. 

Choose e^ = 1 

J 

1 y w.A. > T (10) 

. L 11 
1=1 

Otherwise, choose e^ = 0 

where the terms have the same meaning as before. 

g 

Compared with the binary feedback, the feedback of the Cj in the analog 
shift register is more involved. Here three types of feedback were explored. 

3 

(1) Soft feedback. The original cj computed from the received bits are 

circulated in the register. This assumes that the error probability of the 
corresponding bit in the binary shift register has not changed. (This would 
have been a reasonable feedback connection if it had been used with the dotted 
feedback connection (Fig. 32 of appendix B) for the binary shift register. 
However, soft feedback was explored in conjunction with the corrected binary 
feedback, since this technique had been proposed in reference 1.) 

(2) Hard decision feedback. After a bit is decoded its error probability 

3 

is assumed to be zero, which means that the corresponding Cj is zero. This 
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causes a computing difficulty toward the end of each decoded block where the 
weights approach infinity. This difficulty was overcome by the rule of assign- 


ing to the decoded 


c g 

3 


a number corresponding to a low error probability. 


(3) Full APP feedback. The name implies that after each bit is decoded, 

g 

the bit error probability after decoding is fed back as its corresponding c ■ . 

3 3 

If one assumes bit for bit independence, the c. can be calculated as follows: 
Let 3 


J 

X = l W A - T = - log [p/ ( 1 - p)] (U) 

j = l 3 3 


where p is the bit error probability after decoding 

e“ x = p/(l - p) 


( 12 ) 


c® = -log(l - 2p) = -log[(l - e _X ) / (1 + e” X ) ] 

However, it should be noted that bit error probability after decoding each bit 
depends strongly on the bit error probability of many other bits and therefore 
the independence between bit error probabilities under which the coding theorem 
has been developed does not strictly hold. Nevertheless, this decoding method 
will be shown to perform somewhat better than the two others described 
previously. 


MEASURES OF PERFORMANCE 


In order to compare the effectiveness of different coding schemes, meaning- 
ful measures of performance must be found. The normalizing assumptions as well 
as the calculation of the performance measures and curves are shown in appen- 
dix D. Here the shortcomings and advantages of the various performance mea- 
sures will be discussed, and the justification for the ones chosen will be 
given. One measure is the output bit error probability for a given signal 
energy per bit per noise spectral density, E^/N Q . This measure accounts for 
the rate loss of codes with various ratios of information to parity bits. How- 
ever, it does not account for the fact that various coding schemes will yield 
different output error streams with different statistical properties, the rela- 
tive desirability of which will vary according to the processing strategy of 
the decoded data that follows. 

Scientific data from space probes are generally not sent in units of bits 
but words (groups of bits). For comparison of different coding schemes, word 
error probability will therefore be chosen. Some information about the perfor- 
mance of certain block codes has been published in terms of word error proba- 
bility, with a word length equal to the block length of the code, but only 
because the bit error probability could not readily be calculated. The mea- 
sure based on block length does not permit direct comparison of codes of 
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various block lengths. For convolutional codes with threshold decoding usually 
the probability to the first error pj(e) has been computed. This measure is 
not satisfactory for the comparison between different kinds of codes. When the 
coding and decoding are completely simulated, one can choose any word length. 
The word length chosen for this study is six information bits, since this 
length has been used in the past on scientific deep space probes, such as 
Pioneer VI and VII. For both probes the simplest error detection codes are 
used, that is, one parity check bit is added to each six-bit data word. In 
such a code, words known to contain errors are deleted. For a space-to-ground 
telemetry channel, scientific data resulting from measurement of physical quan- 
tities are transmitted at specified intervals. As long as the word deletion 
rate is small (in the order of a few percent), the random deletions may have 
little more effect on the usefulness of the data than a coding rate loss. If 
one concedes this, the coding gain for the seven-bit parity check code is shown 
in figure 3. The coding gain is greater than 2.5 dB over a surprisingly large 
range of signal-to-noise ratios. One must keep in mind, however, that the 
deleted words in error occur at random. Random deletion of words may have a 
much more serious effect than the dB rate loss equation (10 log ^ (1-deletion 
rate)) would suggest, since for many experiments, groups of words are needed. 
(For an 8-percent deletion rate the rate loss is only 0.37 dB.) Thus, if 
another simple code with no deletions had the same word error probability, it 
might be preferred. Allowable deletion rates are somewhat subjective and 
depend on the experiment to which the data belong. The cost in deleted words 
(see fig. 3) was not treated as rate loss but instead was marked on the curves 
as a parameter. 


DESCRIPTION OF DECODING SIMULATIONS 


Most of the experiments were performed entirely on a general purpose 
computer for the following reasons: First, the gaussian channel is easily 

simulated on the computer. Second, decoding equipment for the APP decoding may 
best be simulated by a computer. Third, the input to the decoder and the out- 
put from it are available to the computer for immediate detailed comparison and 
analysis. Fourth, for low bit error probabilities, simulation of the data 
stream that includes errors allows a computing economy if only potential error- 
causing situations are generated, and the resulting output is decoded and 
analyzed. This is explained in detail in appendix E. 

The overall experimental procedure for block code analysis is outlined in 
the accompanying flow chart. The flow chart has been somewhat idealized so 
that the arrangement would automatically generate sufficient data to produce 
the complete analysis for a given code at a specific input bit error probabil- 
ity. This would not have been difficult to program as an executive routine; 
however, the rather unpredictable computer time required to generate suffi- 
cient statistics made this method impractical. 

Some of the individual blocks identified by reference numbers on the flow 
chart will now be discussed. 
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Store error location in the block divided into seven-bit intervals and accumulate 
totals so that the following table is generated : 


Total number of words with a given number of errors 

Number of First Second Z ... . 

errors in seven-bit seven-bit ... num 

a word word word of errors 


Total number 
of error words 
in the — — 


First 
seven -bit 
word 


Second 

seven-bit 

word 


Total number 
of error 
words 


Abstract further information resulting in the following table : 


Number of errors 

Total number of correction pulses when 

in the block 

decoding information and parity bits 
012345678.. . 

0 

1 


1 

2 

Number of words with a given number of 

3 

correction pulses and output errors 


(Results are printed after each run) 


End of one 
run (NW words 
generated) 


Calculate average number of output errors per block and its square for 
statistical evaluation of the fluctuations of the data 


/ End of runs (NR ^ 
runs with same number of 
\input errors generated) ^ 


Start another 
run 


Combine tables 1 and D having identical information for the larger number of 
words (equals words/run x number of runs 

Results ore printed after each specified number of runs as well as after the 
final run 


Calculate the confidence interval of the average number of errors per block 




Another run (increase NR) or longer runs 

_ {increase N) to get an average of at least 

four output errors/run in order to have a 
meaningful t test 

T 

Find analysis of the code performance for the given number of input errors 
per block 

Besides the table mentioned above, the following information is printed: 

Histogram of the number of output errors in information bits against the 
number of blocks with a given number of output errors in the 
information bits 

Number of blocks with nonzero information errors 

Total number of correction pulses 

Totol number of errors corrected correctly 

Total number of correction pulses in parity 

Total number of errors in information not acted upon 

Total number of input error bits in the seven -bit words 

Sum of the correct land D levels 

Sum of error land 0 levels 

A two-dimensional histogram of the number of correction pulses (horizontal) 
versus number of information output errors (vertical ) versus total number 
of words with a given number of correction pulses and a given number of 
errors in the information bits 


Is the error 

No :ontnbution of this series^^Yes 

\of runs small compared^^ | 
^\to the last series?^^ 


t 


Change number of errors in per block 
to Ntl and start a new series of runs 


Final code analysis: 

Calculate bit error probability, block error probability, six-bit word error 
probability, six- bit plus parity word error probability, six-bit plus parity deletion 
rate (odd number of errors in the word), and seven-bit word error probability 
in relation to the word position in the block 

" I 



Goto beginning for next 
bit error probability 



(1) As mentioned in the Theory of Operation section the code performances 
depend only on the error locations and not on the sequence of zeros and ones 
representing the information. For this reason an all 0 information bit 
sequence was used. Errors are then easily identified by ones. 

(2) , (3) The method of generating errors has been described elsewhere 
(ref. 2) . 

(4) This information is collected to give clues as to how codes fail. 

(5) This is an examination of the error distribution throughout the 
block . 

(6) This is a test of the correlation between number of output errors and 
number of correction pulses. If a large number of correction pulses occur, 
this implies there are errors in the block and this block could be deleted. 

(7) , (8) , (9) These were hand simulated. The activity described in 
these blocks serves the purpose of obtaining sufficiently long data runs. The 
total run is partitioned so that a meaningful t-test can be applied, and so 
that the 95-percent confidence interval is not greater than ±1/2 of the mea- 
sured parameter for potential error-causing situations of significant probabil- 
ity of occurrence. This interval might seem large, but when all experiments 
are combined (with different numbers of errors per block and different input 
error probabilities), statistical errors tend to cancel. Also, knowing the 
average error rates within a factor of 2 is quite adequate when one considers 
how small a change of signal-to-noise ratio changes the error rate by a factor 
of 2. 


(10) For each type of potential error-causing situation, a summary of the 
performance is printed. In addition, the input bit stream is analyzed to 
assure that the overall distribution of levels is truly gaussian. 

(11) Significant information is added together, weighted by the probabil- 
ity of occurrence of the different output error-causing events. 

The flow chart for the decoding simulation of the convolutional code is 
similar except that it also contains other tabulations of interest which will 
be discussed in the section on test results. For high error probabilities it 
is more efficient and accurate to simulate a gaussian channel directly rather 
than use only potential error-causing events. Data points obtained in this 
manner have been shown as solid symbols on the appropriate figures. 


TEST RESULTS FROM HARDWARE ENCODING/DECODING SYSTEMS 


A hardware model of the (24,12) majority equation decoder was used to 
determine the performance of this type of coding scheme on the binary symmetric 
channel. The test simulated this channel. Data consisted of a pseudo random 
sequence generator, and errors were introduced randomly by synchronously gating 
a threshold detector from a gaussian noise source. 
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Experimental data have been obtained on this code and plotted in 
figure 4. Channel bit error probability, P e , is plotted against the output 
error probability from the decoder. Each point represents a sample size of 
500,000 bits. For the (24,12) double error-correcting code, for example, a 
channel error probability, P e = 5xi0" 3 , gives a decoded output bit error rate 

of P bit = 10 -5 . Error correction performance from a mechanized (73,45) block 
code has been obtained and plotted in figure 5. The curves in figures 4 and 5, 
however, are drawn from the computer simulation data to show the excellent 
agreement . 


TEST RESULTS FROM THE COMPUTER EXPERIMENT 


The test results are presented and discussed in two stages. First, the 
results for the individual codes are shown and second, the codes are compared 
to each other and to no coding. For ease of comparison, the second step neces- 
sitates superimposing some of the performance curves already given. 


Performance of the (15,7) Block Code 

Figures 6 and 7 represent the raw data for the code. These figures show 
on a bit and on a word error probability basis that APP decoding has a large 
error reduction capability beyond that of unweighted (majority decision) decod- 
ing, the reason being that APP decoding corrects many of the otherwise error- 
causing situations. On the other hand, while majority decoding always corrects 
two errors in the block, APP decoding will sometimes make output errors in such 
cases, which accounts for as many as 20 percent of the errors made at an input 
error probability of 3 percent. For higher error probabilities, even more of 
the double error words will be incorrectly decoded, which means that eventually 
the curves for APP and majority decoding intersect. However, this happens at 
an error rate at which neither decoding scheme is of any use for this code. 

The curves have been translated into plots of error rates versus E^/Nq in dB. 
Figure 8 shows that majority decoding has a very limited gain of only 1 dB at 
a bit error probability of 10’ 5 , and the coding gain reduces rapidly as 
E^/N 0 decreases. APP decoding tends to keep the coding gain constant over the 
region of interest, with a gain of about 2.6 dB over no coding. Note that 
there is little difference in performance between full APP and hard decision 
feedback decoding. 

Since the simple seventh bit parity check code seems to work so well when 
a small deletion rate is permitted, its effect when it is concatenated with the 
different codes was investigated. As figure 9 shows, the performance, which 
includes the rate loss of 0.67 dB, is slightly worse by 0 . 2 dB. It will be 
seen that this condition prevails for all the codes tested. The data are 
included since it might be useful to employ the seventh bit parity check code 
before further encoding in a spacecraft, so that if the encoder should fail it 
could be switched out, while the parity check would continue to make the 
received data useful. Deletion rate with coding is negligible compared to that 
of the seventh bit parity check code, and therefore is not shown. While hard 
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decision decoding performs slightly better than full APP when the parity check 
code is not included, the situation is reversed when it is included. This is 
explained by the fact that compared to hard decision feedback, full APP gener- 
ates significantly more single error words while producing fewer words with 
more than one error. 

It will be remembered that in the cyclic block codes, it is possible to 
decode the parity bits first, and then the information bits. This would be a 
useful technique for the full APP decoding, since the word error probability 
for the second seven-bit word is about half that of the first word. The hard 
decision feedback decoding has the same word error probability for the two 
seven-bit words, and it would be wasteful of computer time to decode the parity 
check bits also. 


Performance of the (73,45) Block Code 

The raw data and the data converted to the performance curves are shown 
in figures 10 through 13. Most of the remarks made for the (15,7) apply also 
to the (73,45) code, except that the performance of the latter is considerably 
better. There is also a clear advantage compared to the seventh bit parity 
check code, at least as far as APP decoding is concerned. For the 3-percent 
error probability data, the full APP decoding is clearly superior to hard deci- 
sion APP decoding. It was, therefore, the only method used for the 1-percent 
runs since they must be longer than the 3-percent runs to get statistically 
significant data. 

Figure 14 shows the word error probability normalized to the first word as 
a function of the word position in the block for various decoding systems. The 
figure illustrates the effect of the different types of feedback. Data points 
for majority decision decoding in which the original bit decision is fed back 
(dotted connection in figs. 31(b) and 31(c) in appendix A) are not shown since 
they were not measured. However, it is clear that the corresponding curve 
(fig. 14) should be the horizontal line I, since the word position in the 
block will not affect any coding decisions. Curve II shows the beneficial 
effect of hard decision feedback on majority decoding. This effect becomes 
even more pronounced for APP decoding with hard decision feedback (curve III). 
But the improvement levels off rapidly. Curve IV shows full APP decoding for 
identical data. As expected, it begins in a manner similar to the one for hard 
decision APP feedback, but the improvement continues all through the block. 
Consequently, it is worthwhile to decode the parity check bits first and then 
the information bits. Figures 12 and 13 give the performance for decoding all 
73 bits. The additional improvement from the extra effort of decoding the 
parity bits first can be estimated from figure 14. 


Performance of the (24,12) Convolutional Code 

The raw data and the data converted to performance curves are shown in 
figures 15 through 18. In spite of its higher error correction capability the 
performance of the (24,12) convolutional code is not much better than that of 
the (15,7) block code for majority logic decoding. Figure 19 shows some of the 
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details of the error dispersion. The decoder slides into a bit stream contain- 
ing a potential error-causing situation within a constraint length of 12 infor- 
mation and 12 parity bits. Figure 19(a) shows that some output error patterns 
are 35 bits long. A comparison of figures 19(a) and 19(b) shows that the alarm 
indeed reduces the average length of the error patterns. Especially, it 
removes the curious peak at a distance of 14 bits. Even though the alarm 
reduces the average number of errors per potential error-causing situation, the 
actual number of output error packets is increased. For APP decoding the alarm 
is not useful. The dispersion for APP decoding, as shown in figure 19(c), is 
much larger than for majority decision decoding, but is compensated for by the 
much smaller number of output error packets per error-causing situation. 

Another fact worth noting is that the shape of the dispersion curves seems to 
be independent of the number of input errors per potential error-causing 
situation . 


Performance of the (44,22) Convolutional Code 

The raw data and the data converted to performance curves are shown in 
figures 20 to 23. A comparison of figures 13 and 23 shows that for the convo- 
lutional code there is a 0.4 dB difference in performance between concatenating 
the seventh bit parity check code and not concatenating, while for the (73,45) 
block code there is only a very small difference. This is explained by the 
fact that errors occur more in bunches in the (44,22) convolutional code than 
in the (73,45) block code; consequently, the ratio of double error words to 
single error words is relatively high. As far as error dispersion is concerned, 
the remarks made for the (24,12) code hold, except that the average dispersion 
has also increased with the increased constraint length (see fig. 24). 


Summary of Test Results 

The performance of the codes is summarized in figure 25. To illustrate 
the type of results obtained, assume the following condition: The six-bit word 
error rate is not to exceed 10 -5 . Then, for no coding the Eb/^o required is 
about 10.4 dB, while the undetected word error rate for the simple parity check 
code requires only 7.8 dB, which, of course, includes a 4.8x10'^ word deletion 
rate (due to parity tagging). If one considers this deletion rate negligible, 
then, for majority decoding, only the (73,45) code shows a moderate improvement 
of 0.8 dB over the seventh bit parity check code. For APP decoding all codes 
show an improvement: 0.2 dB for the (15,7), 1.0 dB for the (24,12), 1.5 dB for 

the (44,22), and 2.1 dB for the (73,45) code. For all codes investigated, APP 
decoding shows about a 1.5 dB gain over majority decision decoding. 

Figure 26 helps to visualize the error bunching that occurs after decoding. 
The figure is essentially a plot of double error seven-bit words versus single 
error seven-bit words. The steepest curve is that of the binomial channel (no 
coding). The remaining curves show that there are many mory double error words 
after decoding than a binomial distribution of errors would suggest. The 
least error clustering occurs in the (73,45) code, when it is full APP decoded. 
Consequently, except for the (73,45) code, performance is generally degraded 
when the seventh bit parity check code is superimposed on the other codes. 
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ERROR DETECTION FOR BLOCK AND CONVOLUTIONAL CODES 


Error detection methods discussed in this section have been tested only 
sufficiently to prove that they will be of value when an extremely low error 
rate is required with a constant computation rate. 


Suggestions for Further Error Detection for Block Codes 

The (73,45) code was the most powerful investigated. It gave an advantage 
of more than 1 dB over the simple parity check code, with the additional advan- 
tage of no deletion. However, it is legitimate to ask if a still lower error 
probability can be achieved at a cost of deleting a moderate number of words 
likely to be in error. Some preliminary work regarding this question will be 
presented here. 

The most obvious answer is to try to concatenate two codes. The simple 
seven-bit parity check code was concatenated with the four codes investigated 
in this report, and, in general, overall performance decreased slightly. 

Forney (ref. 3) , however, proved that concatenating two powerful codes can 
improve overall coding gain but at the cost of increasing complexity at the 
encoder as well as at the decoder. To hold spacecraft complexity at a minimum, 
it would be desirable to achieve a coding gain at the cost of increasing 
earth-based decoder complexity alone. A simple example was given in refer- 
ence 2, where the trade offs between error rate and deletion rate were empha- 
sized rather than the coding gain, which was small. To repeat the theme, space 
scientists often choose to disregard information with a higher error probabil- 
ity in order to increase the reliability of their data. The same method seems 
applicable to full APP decoding, which leaves, in the Cj register, a sequence 
of bit error estimates after decoding. For several data runs the estimated 
block error probabilities were calculated for each block from Cj . A typical 
result is shown here. In 800 blocks with a potential error-causing situation 
of 7 errors per block, and a 1.5 percent average bit error probability, 83 
error blocks occurred. If a block error probability threshold of 10“ 3 were 
chosen, all the error blocks would have been deleted at the cost of deleting 
50 percent of the blocks with 7 input errors which were correctly decoded. It 
must be remembered that at the 1.5-percent input error probability, only 1 per- 
cent of the blocks have potential error-causing situations. To get an idea of 
the small deletion rate, 1,000 blocks containing 42,000 data words were decoded 
with one input error each. Each of the calculated output block error probabil- 
ities was smaller than 10 -5 , indicating that virtually no words are removed 
from blocks with few input errors. 

The deletion rate for any chosen threshold can be estimated by determining 
the distributions of the word error probabilities, given a certain number of 
output errors. For high input bit error probabilities these distributions can 
be simply obtained by generating the error stream in the more natural way, 
where the number of errors in the block are not predetermined. This was done 
for a data run with a 7-percent bit error probability, which strains the per- 
formance of the nominally four error correcting code. The cumulative distribu- 
tions are shown in figure 27. Unfortunately, the distributions are not 
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sufficiently well approximated by normal distributions, or one could now calcu- 
late deletion versus error rate for all possible thresholds. However, from the 
tabulated output, for a deletion threshold with probability of no error equal 
to 0.4, 99.5 percent of the blocks in error after decoding were deleted at a 
deletion rate of 29.4 percent. If one is willing to accept this deletion rate 
as a simple rate loss, 10 log (1/ (1-deletion rate)) = 1.5 dB, the performance of 
the code would have improved by 3.3 dB. This might well be a useful technique 
of gaining data from space probes when switching to a lower transmission rate 
is not possible, and when the error rate would have otherwise been considered 
unacceptable. 

Another attempt to detect error blocks was somewhat disappointing but pro- 
vided valuable insight into the performance of the decoder. It was noticed 
that blocks in error seemed to fall into two groups, either they had very few 
errors or a large number of errors. It was thought that by applying majority 
decoding after the APP decoding of the complete block, which, of course, would 
eliminate errors in all the blocks with less than four errors, one would easily 
detect blocks containing many errors by the behavior of the sum of the A 
equations for each decoded bit. But the majority decoder treated error blocks, 
which had 10 errors after APP decoding, exactly as if no errors were present; 
that is, each A equation is equal to zero. This means that the decoder 
would accept the error stream exactly as it was. It has therefore been demon- 
strated experimentally, that when the decoder fails, it tends to decode to the 
nearest correct neighbor of the actually transmitted code word. This conclu- 
sion is further strengthened when one observes that 9 and. 11 error sequences 
from the APP decoder are treated as a 1 error sequence by the following major- 
ity decoder. Most of the time the majority decoder will therefore add another 
error to the others in the above-mentioned cases. While the above method is 
not satisfactory for error detection and correction, it would tend to reduce 
the block error probability to one- third of what it was before, and it would 
reduce the bit error probability by about half if it were simply used for 
error correction. 

Combining the two error detection schemes may result in virtually error- 
free decoding. If, from the error probabilities of all bits in a block, one 
calculates the block error probability after decoding and then removes all 
blocks that have a higher error probability than a given threshold, one elimi- 
nates virtually all blocks with large numbers of output errors. The remaining 
blocks are then decoded by the majority decoder, and all remaining error blocks 
with less than four errors are corrected. 


An Error Detection/Deletion Strategy for Convolutional Codes 

The performance of majority decoding of convolutional codes was shown to 
be somewhat disappointing. Because of error clustering and rate loss, concate- 
nation of the single parity check code did not improve the overall performance. 
Hence, a detection/deletion scheme that will take advantage of the error 
clustering is desirable. 
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The scheme tested in our simulation is based on the fact that the S 
equations can be combined in more than one way to result in a set of orthogonal 
equations. For instance, Massey’s A equations for the (24,12) code are as 
follows : 


Ai = Si 


a 2 

= s 7 



a 3 

= S 8 



a 4 

= Sg 



a 5 

— S 2 + 

s 4 

+ S u 

A 6 

= s 5 + 

Sg 

+ S 12 

th 

the set 

of 

equat 


(13) 


decoder (previously discussed) and also in all our computer experiments (see 
eq. (A15)). The important difference between these two sets of equations is 
that the error terms of previously decoded bits (terms with negative subscripts 
in equations (A17) to (A20)) are distributed quite differently among the A 
equations. The decoding/deletion scheme is now as follows: 


Run two decoders with different sets of A equations in parallel. Delete 
all seven-bit words when the outputs from the two sets of decoding equations 
differ by at least one position per word. Delete also the two words preceding 
the first detection of an error. The reason for the last part of the strategem 
is that there is a tendency for decoding errors to occur at the same positions 
at the beginning of an error packet. This is understandable, since fewer error 
terms are involved at the beginning of an error packet. These initial errors 
then cause random generation of further errors. The effectiveness of the last 
part of the strategem is shown by the following examples. 

Since APP decoding is of more interest, the effectiveness of the above 
strategy was tested with the more powerful decoding method. Runs with high 
error probabilities were chosen since these data would be of very limited use 
without deletion. For the (24,12) code at a bit error probability of 1.5 per- 
cent, 12 error words remained after decoding. The deletion strategy caught all 
errors at the cost of deleting 1.5 percent of all words. The cost is high 
because Massey’s orthogonalization by itself results in a higher error rate 
than that of equations (A15) , and a word is deleted, of course, when either 
decoder makes an error. For a 7-percent input error probability there were 
349 error words in 12,000 seven-bit words. With the deletion scheme, only 25 
error words remained at a cost of deleting 16.6 percent of all words. Without 
the last part of the strategem 116 error words would have remained. 


For the (44,22) code only Massey’s orthogonalization was available. 
Therefore, a second set of A equations was developed (appendix A). For an 
input error probability of 5 percent, there were 145 error words in 12,000 data 
words. With the strategem only 3 error words were left after 446 data words 
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were removed (22 error words would have been left without the second part of 
the strategem) . 

Depending on the type of decoder, the above scheme doubles either decoder 
equipment or computer time. A simplified error detection circuit is shown in 
figure 28. Although the circuit has not been evaluated, its performance is 
believed to be similar to or better than that of the parallel decoders. The 
circuit operates best by connecting to threshold element 1 the set of A equa- 
tions, which if used alone would result in the better decoder. This threshold 
element alone has control of complementing the S register when it detects an 
error. Threshold element 2, with its different set of A equations, only 
serves to detect error packets for comparison. Thus, only error packets 
caused by the better decoder will be deleted. The decoder will simply mark 
bits not checked by the comparator circuit and leave the deletion up to the 
user . 


CONCLUDING REMARKS 


The work reported forms a basis for comparing the coding techniques 
evaluated here with other coding and decoding techniques. The decoding methods 
described are characterized by a constant computation rate per decoded bit, 
independent of the channel noise. The optimum decoding strategy for these 
codes, maximum likelihood decoding, has not been investigated, since computa- 
tion time per bit is too large to be practical. 1 For convolutional codes, an 
algorithm for sequential decoding 2 is available, which closely approaches the 
performance of maximum likelihood decoding. However, sequential decoding has 
a variable computation rate, which might make it ineligible for high bit rate 
coding. Sequential decoding of convolutional codes is presently under investi- 
gation with the same restrictions imposed on the codes as on those in the 
present report, namely, short constraint length and high rate. The most power- 
ful of these codes investigated (ref. 6) is a (50,25) code which has a gain 
over the (73,45) APP decoded code of 1.9 dB over a range of word error 
probabilities from 10" 3 * to 10" 5 . 

Ames Research Center 

National Aeronautics and Space Administration 

Moffett Field, Calif., 94035, Sept. 29, 1967 
125-23-02-00-00-21 


1 In maximum likelihood decoding one calculates for each possible message 

the probability that a given received message is indeed that message. Then 

the message with the highest probability of having been sent is chosen. For 
example, the (73,45) code has 2 45 possible messages. For convolutional coding 
one would have to truncate the bit stream, and the number of possible messages 

is 2^, where N is the number of information bits in the truncated bit stream. 
Again the number of possible messages and associated computing time is 
enormous . 

Sequential decoding algorithm is a tree search procedure for seeking the 
highest probability message in an efficient manner by pruning less likely 
branches . 
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APPENDIX A 


MAJORITY DECODING AND APP DECODING OF A (24,12) 
THREE ERROR CORRECTING CONVOLUTIONAL CODE 


Figure 29 shows the structure of the triple error correcting encoder and 
decoder. The encoding operation is the following: Information bits are fed 

into a 12 -stage shift register sequentially, as they come from the source. 

Taps on the shift register feed the most recent bit, i i2 , and the five older 
bits in the register, ij, i 2 , is, is, and i 6 , to a modulo 2 adder (parity 
generator) . The output of this adder is zero if there are an even number of 
ones in its input; its output is one if there are an odd number of ones in its 
input . 

The output of the adder is the parity bit pi 2 , associated with informa- 
tion bit i i2 , and these are both ready for transmission over the channel. 

(In the actual equipment they are transmitted in sequence.) It should be noted 
that it is the most recent information bit that is transmitted, along with its 
parity, so there is negligible delay in the encoder. As soon as i i2 and Pl 2 
are transmitted, i 13 is read into the encoder, all bits shifting one place to 
the right, and Pi 3 = i 2 ®i 3 ©ii+@i 6 ®i 1 3 and ij 3 are ready for transmission. 

Encoding continues in this fashion. 

At the decoder, received information is fed into a duplicate of the 
encoder, but received parity is also fed into the mod 2 adder. The output of 
the adder at this moment is denoted by Si 2 - 

By definition, at the encoder, with modulo 2 addition always understood, 

Pl 2 = i l® i 2 ® i 3 ^ i 5 P i 6 ® i 1 2 (Al) 

Now at the decoder the received forms of these information bits are added to 
the parity bit pi 2 ; therefore, if the bits were correct, pi 2 would be added 
to itself, thus getting zero. If one of the bits were wrong, Si 2 would equal 
one; if two were wrong, Si 2 would be zero, and so on. Therefore Si 2 is 
independent of the actual transmitted values of the seven bits of which it is 
composed and depends only on the number of errors in the received versions of 
those bits, being one if the number of errors is odd, and zero if the number of 
errors is even. 

In order to analyze this precisely, denote the received bits by 

1 i * p iP 

ik = i^e^ and Pj^ = Pk 0e k ( t i ie error term, ek or e^ is zero if the correspond- 
ing bit was received correctly, and one if it was received incorrectly); then 

i i i i i i P 

Si 2 = ei®e 2 ©e^e 5 ©e 6 ©ei 2 ©ei 2 (A2) 

Now, at the moment shown in figure 29, i{ is at the right end of the 
decoder chain, and the function of the decoder at this instant is to decide 
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whether i[ is correct or not, and change it if it is wrong. Another pair, 
i{ 3 and p{ 3 , will then enter the decoder, i 2 will be at the right end, and 
the same process will decode i 2 . Decoding will continue in this fashion. In 
order to show how i[ is decoded, make the preliminary assumption that there 
have been no errors before i{ and pj . 

With this assumption it can be seen that when S x was formed 12 bits ago, 
the only bits involved that might have been wrong were i^ and pj, so that 


51 = et©e? (A3) 

Similarly, up to S 6 

5 2 = e 2 ©e]> (A4) 

5 3 = e|©e^ (A5) 

54 = e4@e^ (A6) 

5 5 = e5®e^ (A7) 

5 6 = e£ffie£ (A8) 


When the next bit enters the shift register, ii is in position 7 of the 
register and will be added modulo 2 to the seventh parity and information bits. 
Thus , 



s 7 

1 P 1 
- ey©ey©e^ 

(A9) 

Again all bits shift by 

one 

and the modulo 2 addition gives 
i p i i 

(A10) 


s 8 

= e 8 ©eh©e 1 @e 2 

Similarly, 


s 9 

i p i i 
= e9©e§©e 2 ©e 3 

(All) 


s i 0 

= ei o©e?o© e l® e 3® e 4 

(A12) 


S 11 

= 61 i® e^BeJ® 

(A13) 

and as in equation (A2) 


S 12 

- ei 2 © e^ 2 © ei©e 2 ©e 3 ©e3©e 8 

(A14) 


Another way of seeing how the S equations are developed is to assume 
that ii has been correctly decoded into i°, which means that the value of 
ej- is known . ? 

ej = 0 if ii = ii correctly received 


i 

e l 


1 if i { ^ ii 


error received 
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Therefore, Sx2 can be complemented if e\ = 1 to remove the effect of e} . 

In any case, 

S 12 = e^ee^ee^ee^eegeei 2® e ?2 

Now the S equation register is advanced one step to the right, thus lowering 
the indices by 1 

0 i i i i i p 

S i \ — e i©e2©e4©e 5© 6 x x®^l 1 

Consecutively removing the term ex (if it is present in the S equation) and 
shifting down by 1 will generate the remaining S equations. 


The 12 S equations are now combined to give the following set of 
equations : 

Ax = S x = e x©e^ 

A2 = S2 + Sg = ex©sg©e2®c^ 


(A15) 


A3 = S7 = ex©e 7 ©e^ 

A4 = Sg + Sx2 = ® x©c 5©eg©e g©e x 2®c^©e^2 

A5 = S4 + Sxo = e x® e 3©ex o® e ?® e ?0 

Ag = S3 + Six = 6 X®^2©c 4©e x x©c^©e^ x 

To simplify the notation for APP decoding, the above set of equations is 
written in abbreviated form as 


n. 

1 


A = ei© l 

j = l J 


(A16) 


where n^ is the number of error terms in the given equation minus one. All 
addition is modulo 2. The important thing to note about this set of equations 
is that it is "orthogonal in ix"; that is, e\ occurs in each equation, and no 
other term occurs more than once in the whole array. This orthogonality permits 
use of the following decoding rule to determine e}; that is, to determine if 
i x is right or wrong. If more than three of the six equations are equal to 1 3 
then ix is wrong (e\ = 1) and must be changed . 

It will be shown that this rule will decode i{ correctly if no more than 

3 of the 24 bits ix- . . Px2 are wrong. (In fact, only 22 bits are con- 

cerned, since p 3 and p^ are not in the equations.) To verify that the rule 
works is simple. If ix is right and one, two or three other bits are wrong, 
there can be at most 3 ones among the equations, since each of the other bits 
occurs in only one place. But 3 or less ones is not enough to change i{ . If 

i x is wrong, it makes all six equations one, and if i{ and another bit or 

two are wrong, the other bits restore one or two of the equations to zero but 
leave at least 4 ones. Each of these cases gives more than 3 ones and thus 
leads to correcting ix- 
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Thus i[ will be decoded correctly if not more than 3 of the 22 bits are 
wrong. Actually some combinations of more than 3 errors will also be decoded 
correctly. Now, when it is decided that i{ is wrong, complement it and the 
values of S 7 , S 8 , S 9 , S i0 , $ii> and S 12 * This removes the ei term. Thus, 
before a new pair of bits is entered, the effect of the i{ error has been 
eliminated, and for decoding i 2 , the decoder is in the same state it would 
have been in if i{ had been correct. As far as the decoder is concerned, 
there have been no previous errors . 

The S register resetting operation is called hard decision feedback. 
Assume that the decoded bit is always decoded correctly. If e^ is called 
the possible error in the output bit, then Si 2 after decoding should be 
rewritten as 



0 Olllll P 

S 1 2 = e l© e 2® e 3® e 5© e 6® e 1 2® e 1 2 

(A17) 

and shifting by one 

space, Sn actually becomes 



„ oiiiii p 

Sll _ e o® e l® e 2® e 4® e ^ e i i©ey 1 

(A18) 

likewise , 

c- 0 _ 0 i i i_ i p 

Sjq - e_ 1 ©eo©ei©e3©e4©e 1 o© e i 0 

(A19) 

similarly down to 

Si : 



r 0 ooooip 

s l = e ^i Q©e_g©e_g©e_g©e_g©ei©eY 

(A20) 


In other words, the S equations actually depend on previously decoded bits. 
This explains why once a decoding error is made, more errors tend to be made 
even though only correct bits may subsequently enter the decoder. 

Figure 30 shows the APP decoder which includes an analog circuit for com- 
puting the weighting factors. The weighting factors for the gaussian channel 
are computed as follows: The incoming voltage levels V for each bit are 

passed through a nonlinear amplifier that has an output 

-log e { c °th [ (V/a 2 ) * | V | ] } where V and a 2 are the average received voltage 
and its variance. The top analog shift register has as inputs the set of c^ 
and the bottom one has as inputs the set of cP. The kth analog adder above 
the analog shift registers has as inputs the set of c r s to form C^. The out- 
put of this adder is fed to a nonlinear device that has an output of 
2 log e [coth(x/2)] for an input of x. This is the weighting factor for the 
parity check A^. The threshold T is formed by taking half the sum of the 
weighting factors as called for by equation (8) . Since the analog circuit 
computes the correct set of weights and the threshold T at any instant, it is 
combined with the decoder of figure 29 to give a complete APP decoding circuit. 
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APPENDIX B 


MAJORITY DECODING AND APP THRESHOLD DECODING OF THE 
(15,7) BOSE - CHAUDHUR I CODE 


Decoding the (15,7) Bose-Chaudhuri code will be described as an example 
of threshold decoding of a block code. This is a type of cyclical block code 
that corrects two errors in a block. Figure 31 shows the structures of the 
encoder and the decoder. The encoding operation is as follows. Information 
bits are loaded in parallel into a seven-stage shift register. Taps on the 
shift register feed the most recent bit i^, along with i 5 and i 7 into a 
modulo 2 adder, and the output is transmitted as well as fed back into position 
7 of the shift register. The shifting and transmission continue through infor- 
mation bit i 7 . The process is continued until eight parity bits are 
generated, namely, 

i k = V^k-S^k-y’ k = 8, 9, .... 15 (Bl) 

After all 15 code digits have been shifted out, the 7-stage register once again 
contains the original information, ii to i 7 . 

At the decoder, received information is fed in parallel into a 15-stage 
shift register. Taps lead to four mod 2 adders to calculate the parity equa- 
tions. Majority decoding, shown in figure 31(b), is ordinarily used. The 
equation 

. i . t . f . r 
A x = i 5 ©i 7 ©i 8 


can be reduced to the transmitted information plus error terms 

Aj - i 8 ©e 5©i 7 ©e 7 ©i 8 ©e 3 


but from equation (Bl) 
thus 


i 1 - i 5 ©i 7 ©i 8 


A 2 = i i 5©e 7 ©e 3 


Writing all parity equations in the same manner results in 

Aq = ii©e x 
A | = ii©e 5 ©e 7 ©e3 
A2 - i 1 ©e 3 ©e 4 ©e \ 2 

A3 = i^e^ex (Pe li+ 

* f 

A4 — 1 ]©eg©ei 3©6 ;l 5 


(B2) 
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The majority decision rule is i° = 1 if 

4 

I A i 1 3 (B3) 

0 

To produce a threshold decoder which performs exactly like the majority decoder 
(see fig. 31(c)), connections from ±i are added. The first parity equation 
for this decoder becomes 


A ^ — A^Si^Se^ 

= i i©e5©e 7 ©e8©i i©ei 


= ei©e 5 ©e7©e9 


Similarly, 


Ai = ei©e 5 ©e7©e 8 
A 2 = e 1 ©e3©e4©ex2 
A 3 = ex©e2©ex o©ex 4 
A 4 — e x©Gg©e x 3©c x 5 


(B 4 ) 


The A^ are orthogonal in ex and are independent of the value of the trans- 
mitted bit. If there are an odd number of errors in the equation Aj = 1; and 
if there are an even number, A^ = 0. The decision rule with these parity check 
equations is : 


If 


4 


I A i L 3 
i=l 


(B5) 


then ij is assumed to be in error; hence, change it to produce i°, the out- 
put bit, which still may contain an error (i ^ ^ ix). 

This decision rule is of exactly the same form as that for the convolu- 
tional code discussed earlier. If all the received bits were correct, each 
equation for A^ would add to 0. If one or two of these bits were wrong, at 
most two of the Aj_ would equal 1, the threshold would not be exceeded, and 
the bit would be correctly decoded. 

After decoding the first bit i x , the bit is corrected and circulated to 
position 15. Identical decoding rules are applied to decode i^. The process 
continues until all information bits are decoded. The formal proof for the 
above is given by W. H. Peterson (ref. 4). However, it is simple to advance 
the indices of the parity equations in steps of 1 (mod 16) and to show by sub- 
stitution that the equations always form orthogonal parity checks for the bit 
which is presently in position "one" in the decoder. Even the parity bits can 
be decoded in this manner by shifting a full cycle, a fact which is shown to be 
useful in the discussion of the test results. 
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Figures 31(b) and 31(c) show dashed and solid feedback lines. It is not 
at all clear which connection should result in the better performance. 

Clearly, when the number of errors in the block does not exceed the error cor- 
rection capability, the decoder will eliminate all errors with both types of 
feedback. For potential error-causing situations, it was found that the feed- 
back of the decoded value will result in a lower average error probability. 

This is the only feedback connection of the binary shift register explored in 
connection with APP decoding and will be. called binary hard decision feedback. 
The APP threshold decoder is shown in figure 32. The operation of this circuit 
is so similar to that in figure 30 that no detailed explanation should be 
necessary. 


For completeness, the encoding and decoding equations for the (73,45) code 
are given. Encoding: 


i. = i 1 , ©i. , ©i. 0 ©i. rtl ©i. „ ©i, , ©i. ©i. „ 

k k-4 5 k-43 k- 3 5 k-2l k-20 k-16 k-9 k-3 


where 


i = 46, 47, . . . , 73 


and decoding: 


A} 

A£ 

A3 

A4 

As 

a4 

A4 

a4 

a4 


a{ o 


i 3 ©i i l®i2 5®^26®^- 3 0®i 3 7 ©i 4 3 ® i 4 6 
i 9 ©i 2 3 ® i 24®i 28®-^- 3 5®i 4 l®i 44®i 72 
i 1 5®i 1 6®i 2 0®i 2 7®i 3 3® i 3 6 ® i 6 4®i 6 6 
i 2®i 6®i { 3®il9®i22 ei 5 0 ei 52®i 60 
i5ffii| 2®il 8®^2 1®^ 4 9®i 5 l®i 59®-*- 7 3 
i 8©i 1 4®i 1 7®i 4 5®i 4 7®i 5 5® i 6 9® i 7 0 
i 7®i i 0©i 3 8©i 4 0® i 4 8 ©i 6 2®i 6 3®i 6 7 
i 4©i 3 2©i 3 4®i 4 2©i 5 6 ®i 5 7©i 6 l©i 6 8 
3-2 9®i 3 1 ®i 3 9®i 5 3® i 5 4®i 5 8 ®i 6 5®i 7 1 


(B6) 


(B7) 


The decision rule with these parity check equations is: 


If 


9 


I Ai > 5 
i=l 


. i 

il 


is assumed to be in error; hence change it to produce 


. o 

n ■ 
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APPENDIX C 


DERIVATION OF THE APP DECODING ALGORITHM 


For optimum threshold decoding the rule is chosen to make the conditional 
probability that e m is either 1 or 0 (i^ is in error or not) a maximum given 
the set of J parity checks . 

Pr (e m =V/{Ai>) maximum (Cla) 

In other words, choose e m = 1 . Then 

Pr(e m =l/{A i }) > Pr(e m =0/{A i }) (C2) 

From Baye*s rule, equation (Cla) is rewritten 

Pr( e m =V/ {Ai» = [Pr({Ai}/e m =V) • Pr(e m =V)]/Pr({Ai)) (C3) 

From the orthogonality on e m of the A^ and the digit to digit independence 

J 

Pr ({Ai }/ e m =V) = [1 PrCAi/e^V) (C4) 

i=l 

When equations (C3) and (C4) are substituted into (C2) and the common term 
Pr({A^}) is cancelled: 

J J 

Pr(e m =l) 77 Pr(Ai/e m =l) > Pr(e m =0) 77 Pr(Ai/e m =0) 


J 

77 Pr (Ai/ e m =l)/Pr (A-^/ e in =0) > Pr (e m =0)/Pr (e m =l) (Clb) 

i=l 

Define 

Pr(e m =l) = p 0 = 1 - q Q - error probability of bit e m (decoded bit) 


p. = 1 - qi = probability (odd number of l's among the noise 
bits that are checked by Ai, exclusive of e m ) 


then 


Pr(Ai=0/e m =l) 

= Pr(A i =l/e m =0) = p i 

(C5) 

Pr(A i =l/e m =l) 

= Pr(A i =0/e m =0) = qi 

(C6) 
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Since A i is either 1 or 0 each factor in equation (Clb) is in one of two 
forms : 


If Ai = 1 
If A x = 0 


2A- -1 

Pr(Ai=l/e m =l)/Pr(Ai=l/e m =0) = qi/pi = (qj/Pi) 1 

?A • -1 

Pr(A i =0/e m =l)/Pr(A i =0/e m =0) = Pj/qi = (qj/Pi) x 


(C7) 


CCS) 


Equations (C7) and (C8) are used to rewrite (Clb) as 


J 

77 

i=l 


Cqi/Pi) 


2Ai-l 


> %/Po 


or 


J 2A- J 

77 (qi/Pi) 1 > n qi/Pi 

i=l i=o 


(Clc) 


Since the 


e i 


are independent random variables with 


Pr 



it can be shown that (see ref. 1) 



Pi 




(C9) 


where n^ is the number of error terms in exclusive of e m . Note that 

the product does not include a term for the encoded bit e m ; t also note that 
equation (C9) is also true for p G = (1/2) [l - (l - 2 Yl)] = Yi • Now 


Pi 



Since summation is preferable to multiplication, define: 

Cj = -loge(l - 2yj) 

qi 


Then 


TLa 


= coth j 4 £ C i 

Pi l 2 j=i 3 


(CIO) 


(cii) 

(C12) 
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and 


= coth[(l/2)C}] 

P 0 

Taking logarithms on both sides of equation (Clc) 

i v q i 

I 2Ai log e -1 > l log e — 

i=l p i i=0 p i 

and using equations (C12) and (C13) yields 


I 2Ai log 

j = l 


n i 


coth j I C 

j = i 




log 


coth(— C\ 


+ I 2 log 

j = l 


coth| -j 

j 


If the weights are defined, the threshold equation becomes 


l Mi > I I "i 

i=i i=o 


Further, if 


t = i.i -i 

1 = 0 

the final form of the APP threshold decoding equation becomes 

I > T 


(C13) 


(Cld) 



(Cle) 


(Clf) 
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APPENDIX D 


CALCULATION OF PERFORMANCE MEASURES FROM AN 
ENERGY EFFICIENCY STANDPOINT 


To compare different coding schemes consistent measures of performance 
must be found, and finding them requires first a definition of the channel 
under consideration. The following normalizing assumptions are thought to be 
reasonable for interplanetary channels in the S-band, where the dominant 
source of noise is the receiver, and where provision for bit rate changes must 
be made because of the changing distance between the spacecraft and earth. 

(1) An identical modulator, transmitter, and receiver system will be 
assumed for all coding schemes considered. (Thus rate 1/2 codes will not be 
penalized for their greater bandwidth requirement for the same information 
bit rate.) 

(2) The system delivers a stream of bit log likelihood ratios to the 
decoder (matched filter reception); that is, the individual bit error probabil- 
ity can be calculated from the received voltage at the bit decision time. 

(3) The transmission rate of information is assumed constant. 

(4) The noise is additive white gaussian and changes in average power are 
very slow. 

The received signal power varies very slowly with time. Its magnitude at 
any particular time will be denoted as S. The time required to send a single 
bit is T. The energy per transmitted bit is then ST joules. The noise 
power has a spectral density of N Q W/Hz. The total noise power in a band of 
f Hz is proportional to N 0 f. Since the bandwidth is inversely proportional 
to the bit duration, the noise power in the transmission spectrum is also 
proportional to N 0 /T. The noise energy per bit is therefore N 0 , independent 
of the bit duration. A convenient normalized variable for the calculations 
to follow is the signal-to-noise energy ratio per transmitted bit 

E/N 0 = ST/N 0 (Dl) 

It is well known (ref. 5) that under these conditions, the output of the 
matched filter receiver is a gaussian random variable y whose mean is posi- 
tive or negative according to whether a 0 or 1 was sent in the corresponding 
time period. 

Under these conditions the average bit error probability is 


where E/N Q is the linear signal-to-noise power ratio. For reference for 
large E/N 0 this is approximated within 1 percent by 



for ^— > 5 
N o 


(D3) 


The probability that a received bit i! is in error, given its received 
voltage level, is, from figure 33, ^ 



Having characterized the channel, one can now proceed to describe the 
most often used performance criterion for codes, that of coding gain. In 
short, coding gain is the increase of the received power that would be 
required to achieve the same bit error probability for no decoding as for 
coding. 

With the relationship of equation (D4) , a graph can be drawn of bit error 
probability versus E/N 0 (fig. 34) corresponding to the case of no coding. 

In figure 34, the performance of a hypothetical code is also shown. It can be 
seen that the coding gain is reduced as E/N Q becomes smaller, and eventually, 
the code performs worse than no coding. This is an unfortunate characteristic 
particularly of simple codes, where otherwise they would be most useful, they 
are least effective. 

For the above comparison, the code performance curve is drawn in the 
following manner. It is assumed that the decoded bit versus input bit error 
probabilities have somehow been determined (l). 1 Then by means of figure 34 
for a given input bit error probability (1) E/N Q is read (2). 

Since codes characteristically send more than one bit per information 
bit, a rate loss is added (2-3) 

(E/N g ) r = (E/N q ) + 10 log (1/R) (D5) 

where R is the ratio of information bits to information plus parity bits 
transmitted. This is the E/N 0 which would be received by the identical 
communication system and information rate if no coding were used. For this 
increased signal-to-noise ratio the input bit error probability (4) is lower 
than the input bit error probability to the decoder (1). Only if the decoder 
reduces the output bit error probability (5) below this value can one speak of 

lumbers in parentheses refer to encircled numbers on the figure. 
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* 


a coding gain. Above (E/N Q ) R the output bit error probability is entered (5). 
And the horizontal distance between the point just drawn and the curve for no 
coding is the coding gain (3-6) . 

While the bit error probability criterion is sufficient for selecting a 
small number of interesting codes for a given application, detailed simulation 
is required for the final choice. As explained in the text, word error proba- 
bility for six-bit words has been chosen as the measure of performance. 

Figure 35 shows the steps of the calculations for the specific example of the 
(7,6) parity check code. The word error probability curve for no coding is 
drawn by calculating the probability of errors occurring in a six-bit word 
(l-P(OE)) for given bit error probabilities (1). The calculated point (3) is 
then drawn vertically above the E/N 0 (2) for the selected bit error probabil- 
ity. The coding performance is calculated in the following steps. Select a 
bit error probability (1) . Find the corresponding E/N Q (2) and add the rate 
loss of 10 log(7/6) = 0.67 dB (4). The new increased bit error probability is 
found (5) at which the bits will be received from the same transmitter at the 
7/6 higher bit rate. Calculating the probability of an even number of errors 
in a word received at that average bit error probability results in the word 
error probability (6). However, parity tagged words are discarded. Therefore, 
one must calculate the word error probability on the remaining words, which 
increases the word error probability by a factor l/(l-P(odd number of errors)) 
(7). Also, the information rate has decreased by the additional rate loss of 
10 log (1 (1-P (oddE) ) ) (8). However, this small additional loss is not shown on 
the above curve, since E/N Q is the received signal -to-noise energy ratio per 
information bit before decoding; instead, the deletion rate is shown as a 
parameter in the text. From the two curves thus generated, one can determine 
the coding gain for any desired word error probability. 
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APPENDIX E 


BIT ERROR CALCULATIONS FROM ERROR PACKET SIMULATIONS 


For a low bit error probability, block codes and convolutional codes 
correct most of the errors. To obtain a sufficient statistical sample a very 
long bit stream would have to be examined. It would take hours of IBM 7094 
computer time to evaluate even one code at one bit error probability. There- 
fore, a scheme was developed to calculate code performances by examining only 
potential error-causing situations . 

For block codes the method is very simple. The bit stream is divided 
into blocks of a given constraint length (e.g., 73 for the (73,45) code). A 
given number of input errors are introduced randomly into blocks of data and 
the average number of output errors are calculated simply as the weighted 
average 

P bit = 2 e i [P(i)/N bl ] (El) 

i=i min 

where 

Pj^ bit error probability 
i Q constraint length 

i'min minimum number of errors which can give output errors 
P(i) probability of i errors in a block 
N bl block length 

average number of output errors in i c given i input errors 

In practice only a few terms are needed since P(i) decreases rapidly with 
increasing i for the error probabilities under consideration. 

For convolutional codes, the calculations are not so direct. Intuitively, 
it is clear that a convolutional code compared to a block code with identical 
constraint length and identical error-correcting capability will encounter 
error-causing situations more frequently. A model will be developed which is 
sufficiently accurate to predict the average number of error-causing situa- 
tions which are encountered when N information plus parity bits are 
received . 

Consider a sequence of N zeros and ones where the zeros represent cor- 
rectly received and the ones represent error bits. If one counts the number 
of errors contained for each constraint length, a second sequence results. 
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This is shown in an example for a constraint length of 24, where it is 
assumed that all zeros border the sequence which is shown. 

. . . 000000001010000010000001 . . . 

00000000112222223333333444444444332222221111111000000 . . . 

Each number on the last line represents the number of errors within the slid- 
ing constraint length. Neglecting end effects for large N, the second 
sequence has the same number of members as the first, namely, N. The average 
number of times each member of the sequence occurs is 


Ni = P(i)N 


(E2) 


One is interested only in how many times a maximum occurs, representing an 
error-causing situation. One must therefore calculate the average number of 
shifts that one stays inside an error cluster of a given number of errors. 
(In the above example, the 4 appears nine times in sequence.) 


d i 



(E3) 


where d^ is the average number of shifts within a cluster of i errors. 
Equation (E3) can best be understood when it is derived for an example, 

(i =4, i c = 24) . It is assumed that when the error condition is first 
reached the other i-1 error bits are randomly distributed over the con- 
straint length. When the first and last bit are in error (d = 1) the remain- 
ing i-2 errors may be located in any order in the i c -2 bits. There are 

lc 0 2 ways to distribute the errors. When the last bit in error is in the 

' ' / i _3\ 

second position (d = 2) , there are ( J ways to distribute the remaining two 
error bits, etc. Hence, the average distance is 


. if) * if) * if) ♦ • • • ♦ - 0 ) * Hf 

d 4 = = 6.0 

(?) 

where ^e number of ways the i-1 = 3 errors can be distributed 

over the 23 positions. It can be shown by mathematical induction that 
equation (E3) can be reduced to 


d i = i c /i (E4) 

which is the average distance between errors, provided there are i errors 
in i c . When clustering of more than i errors in i c has negligible 
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probability of occurrence compared to i errors in i c , the average total 
number of error-causing situations is from equations (E2) and (E3) : 

Vdi = P(i)N/di (E5) 

When e^ is the average number of output errors in N • R information bits, 
and R is the signaling rate (equal to 1/2 for the codes considered in this 
report) , then 


Average total number of errors 

in N information bits caused = NP(i)e^/Rd i (E6) 

by error bursts of length i 

As the example shows, a peak is reached in steps of one. Hence, when the peak 
is of magnitude i on the average, it is flanked by 2d^ (i-1). These must be 
subtracted to count peaks of magnitude i-1. 

Ni.i = [P (i-l)N - 2NP(i)]/d i _ 1 (E7) 


and 


Average number of errors in 

N information bits caused by = (N [P (i-1 ) - 2P (i) ] /Rd^^ 1 }e^ 
error bursts of length i-1 


For i-2 error bursts one would have to subtract terms containing P(i-l) and 
P(i). However, equation (E8) is sufficiently accurate, since P ( i ) decreases 
rapidly as i increases for the error probabilities considered. It is also 
valid for the highest clustering that may be considered, since P(i+1) = 0 
for i + 1 > i c ; thus, the equation for convolutional codes for bit error 
probability is: 


P 


bit 


1 _ 

R 


I 

i=i 


min 


P(i) 


- 2P (i + 1) ' 
di 


(E9) 


As in equation (El) only the first few terms need to be considered. 
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Data source 



(a) Encoder. 





(b) Decoder. 


Figure 1 . - Block diagrams of rate one-half convolutional encoder and 

threshold decoder. 




V 


5 Probability that a received bit is in 
error, given its voltage level 

= Voltage level of the received bit in 
units of V 


(a) Error probability of a given bit versus voltage received. 



(b) Calculated Cj from the error probability of the bits. 



Cj = XCj 
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(c) Calculation of the weights. 

Figure 2.- Illustration of the calculation of weights. 
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Figure 3.- Performance of parity error detection compared to no coding. 
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Bit error probability 



Figure 4.- Performance of majority decision (24,12) hardware decoder. 
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Channel bit error probability, percent 


Figure 5.- Performance of the majority decision (73,45) hardware decoder 
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Figure 7.- (15,7) block code - six-bit word error rates 
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Figure 9.- (15,7) block code - six-bit word error performance. 
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(73,45) block code - six-bit word error rates. 
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Figure 13.- (73,45) block code - six-bit word error rate performance 



Normalized word error probabilities 



Figure 14.- Normalized seven-bit word error probability as a function 

of the decoding sequence. 
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Figure 16.- (24,12) convolutional code - six-bit word error rates 
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Figure 18.- (24,12) convolutional code - six-bit word error rate performance 




Percent of events with a given 



(a) (24,12) code; majority decision decoded without alarm. 
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D= Number of bits between first and last decoding errors 

(b) (24,12) code; majority decision decoded with alarm. 
Figure 19.- Error dispersion for the (24,12) convolutional code, 
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(c) (24,12) code; APP decoded without alarm. 


Figure 19.- Concluded. 
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Figure 22.- (44,22) convolutional code - bit error rate performance 
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Figure 23.- (44,22) convolutional code - six-bit word error rate performance 
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Figure 24.- Error dispersion for the 


(44,22) 


convolutional code majority decision decoded with alarm. 
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Figure 25.- Performance of four error correction codes with two 

different decoding methods. 
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Figure 26. - Detected versus undetected word errors for several codes 

and decoding methods . 





Calculated probability of 0 errors in a block 

Figure 27.- Distributions of calculated P(OE) for different counted numbers 
of output errors per block (continuously generated error stream at 
p e = 7 percent) . 
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Figure 28.- Simplified output error detection circuit for the (24,12) code. 
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